AD-A230  881 


f  '* 


HRL-TP-88-33 


I  FORCE 


A  CONCEPTUAL  MODEL  OF  THE 
PERFORMANCE  OF  THE  UNITED  STATES 
AIR  FORCE  FLIGHT  TRAINING  PROGRAM 


DT1C 

1EL.ECTE  f 
.JAN  3  1 1991.J 


John  P.  McGarrity,  Capt,  USAF 
Larry  T.  Looper 
R.  Scott  Curtis,  Capt,  USAF 


MANPOWER  AND  PERSONNEL  DIVISION 
Brooks  Air  Force  Base,  Texas  78235-5601 


January  1991 


Approved  for  public  release;  distribution  is  unlimited. 


1  30  03  2 


LABORATORY 


AIR  FORCE  SYSTEMS  COMMAND 
BROOKS  AIR  FORCE  BASE,  TEXAS  78235-5601 


NOTICE 


When  Government  drawings,  specifications,  or  other  data  are  used  for  any  purpose 
other  than  in  connection  with  a  definitely  Government- related  procurement,  the 
United  States  Government  incurs  no  responsibility  or  any  obligation  whatsoever. 
The  fact  that  the  Government  may  have  formulated  or  in  any  way  supplied  the  said 
drawings,  specifications,  or  other  data,  is  not  to  be  regarded  by  implication,  or 
otherwise  in  any  manner  construed,  as  licensing  the  holder,  or  any  other  person 
or  corporation;  or  as  conveying  any  rights  or  permission  to  manufacture,  use,  or 
sell  any  patented  invention  that  may  in  any  way  be  related  thereto. 

The  Public  Affairs  Office  has  reviewed  this  paper,  and  it  is  releasable  to  the  National 
Technical  Information  Service,  where  it  will  be  available  to  the  general  publ'C, 
including  foreign  nationals. 

This  paper  has  been  reviewed  and  is  approved  for  publication. 


WILLIAM  E.  ALLEY,  Technical  Director 
Manpower  and  Personnel  Division 


MICHAEL  W.  BIRDLEBOUGH,  Colonel,  USAF 
Chief,  Manpower  and  Personnel  Division 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  0704-0188 

?jb*»c  'e porting  Darden  for  this  coil  Meson  o*  information  it  atbmatad  to  avaraga  ^  hour  par  response.  including  t*e  nmt  for  reviewing  nstrucbons.  searching  existing  data  source*  gathering 
and  maintaining  ma  data  needed,  and  completing  and  reviewing  the  collection  erf  information  Send  command  regarding  this  burden  estimate  or  any  other  aspect  erf  the  collection  erf 
information  including isuoaeetlon*  tor  reducmgtht*  burden,  to  Washington  Headquarter*  Service*  Directorate  tor  information  Operation*  and  Report*.  1213  Jefferson  Davie  H,ghway  Suite 
'  20-4  Arlington  VA  22202-4002.  and  to  the  Office  of  Management  and  Budget  ^aperworx  Reduction  Project  10704-0188).  Washington.  DC  20503 

1.  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

January  1 991  Final  Technical  Paper 

4.  TITLE  AND  SUBTITLE 

A  Conceptual  Model  of  the  Performance  of  the  United  States  Air  Force  Flight 
Training  Program 

S.  FUNDING  NUMBERS 

PE  -  62205F 

PR  -  7719 

TA  -  20 

WU  -  07 

6.  AUTHOR(S) 

John  P.  McGarrity 

Larry  T.  Looper 

R.  Scott  Curtis 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Manpower  and  Personnel  Division 

Air  Force  Human  Resources  Laboratory 

Brooks  Air  Force  Base,  Texas  78235-5601 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

AFHRL-TP-88-33 

9.  SPONSORING/MONITORING  AGENCY  NAMES(S)  AND  ADORESS(ES) 

10.  SPONSORING/MONITORING  AGENCY 
REPORT  NUMBER 

11.  SUPPLEMENTARY  NOTES 

12a.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 

12b.  DISTRIBUTION  CODE 

13.  ABSTRACT  (Maximum  200  words) 


The  purpose  of  this  paper  is  to  present  a  conceptual  approach  for  measuring  the  program  performance  of 
the  Air  Force  Flight  Training  Program  (FTP).  The  paper  is  divided  into  five  sections:  The  introduction  provides 
the  background  information  on  four  different  FTP  options  addressed  by  Stoker,  Hunter,  Kantor,  Quebe,  and 
Siem  (1987)  and  establishes  the  general  construction  of  the  program  performance  model;  the  second  section 
addresses  the  effectiveness  of  the  FTP  in  relation  to  graduation  rates;  the  efficiency  section  develops  the  cost 
for  each  of  the  FTP  options;  the  fourth  section,  on  performance,  presents  a  way  to  compare  the  efficiency  and 
the  effectiveness  of  each  program;  and  the  concluding  section  provides  a  summary  of  the  benefits  to  be  derived 
from  use  of  one  of  the  four  options. 


t4.  SUBJECT  TERMS 

cost  M/  Flight  Screening  Program  4/ training 

effectiveness  program  benefits  u  Undergraduate  Pilot  Training'' 

efficiency  selection  ^ 

IS.  NUMBER  OF  PAGES 

14 

IS.  PRICE  CODE 

17.  SECURITY  CLASSIFICATION 
OF  REPORT 

Unclassified 

IS.  SECURITY  CLASSIFICATION 
OF  THIS  PAGE 

Unclassified 

IS.  SECURITY  CLASSIFICATION 
OF  ABSTRACT 

Unclassified 

20.  LIMITATION  OF  ABSTRACT 

UL 

Standard  Form  298  (Apt  2-80) 
Praacrtbad  by  AMM  9*1  Z39- 1 8 
298-102 


N8N  794041 '280-9600 


AFHRL  Technical  Paper  88-33 


January  199i 


A  CONCEPTUAL  MODEL  OF  THE 
PERFORMANCE  OF  THE  UNITED  STATES 
AIR  FORCE  FLIGHT  TRAINING  PROGRAM 


John  P.  McGarrity,  Capt,  USAF 
Larry  T.  Looper 
R.  Scott  Curtis,  Capt,  USAF 


MANPOWER  AND  PERSONNEL  DIVISION 
Brooks  Air  Force  Base,  Texas  78235-5601 


Reviewed  by 


Larry  Looper 

Chief,  Decision  Models  Function 
Manpower  and  Personnel  Division 


Submitted  for  publication  by 


David  E.  Brown,  U  Col,  USAF 
Chief,  Force  Management  Systems  Branch 
Manpower  end  Personnel  Division 


This  publication  is  primarily  a  working  paper.  It  is  published  solely  to  document  work  performed. 


SUMMARY 


This  paper  develops  a  strategy  for  approaching  the  problem  of  measuring  the  performance 
of  a  training  program.  The  performance  of  a  program  consists  of  two  components,  effectiveness 
and  efficiency.  These  two  components  can  be  further  divided  into  the  two  subcomponents  of 
selection  and  training.  This  structure  results  in  four  training  program  measurement  variables, 
training  effectiveness,  selection  effectiveness,  training  efficiency,  and  selection  efficiency.  After 
collecting  measures  on  each  of  the  program  options  for  these  variables,  efficiency  and 
effectiveness  were  related  through  indifference  curve  analysis  to  determine  which  program  would 
provide  the  best  result.  This  combined  measure,  program  performance,  shows  that  a  2C-hour 
Flight  Screening  Program  with  the  Basic  Attributes  Tests  as  a  screening  device  and  the  current 
Undergradu-de  Pilot  Training  program  could  save  the  U.S.  Air  Force  over  $50  million  annually. 


Accession  For 

NTIS  GRAAI 

a r 

DTIC  TAB 

□ 

Unannounced 

□ 

Juet If lcatlon — 

By - 

Distribution/ 


I  Availability  Codea_ 
Avail  and/or 
Special 


Dlst 


t 


PREFACE 


The  present  research  was  accomplished  as  a  cost  comparison  of  four  different  Flight 
Screening  Program  (FSP)  options  that  were  studied  in  the  Air  Force  Human  Resources 
Laboratory  (AFHRL)  paper  TP-86-59.  Flight  Screening  Program  Effects  on  Attrition  in 
Undergraduate  Pilot  Training"  by  Stoker.  Hunter,  Kantor,  Quabe,  and  Siem.  It  is  suggested 
that  the  reader  review  the  Stoker  et  al.  report  before  reading  this  paper  as  it  will  greatly  aid 
in  the  comprehension  of  this  report.  Direction  for  the  study  was  provided  by  Dr.  William  Alley 
of  AFHRL's  Manpower  and  Personnel  Division.  We  also  thank  Mr.  Rich  Hutchins  and  Maj 
Rick  Perry  of  the  Air  Training  Command  for  their  assistance  in  collecting  cost  data  and  Mr. 
Larry  Looper  of  AFHRL  for  his  review  of  this  report. 
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A  CONCEPTUAL  MODEL  OF  THE  PERFORMANCE  OF  THE 
UNITED  STATES  AIR  FORCE  FLIGHT  TRAINING  PROGRAM 


I.  INTRODUCTION 

This  paper  describes  a  conceptual  model  used  to  measure  the  efficiency  and  effectiveness 
of  the  Flight  Screening  Program  (FSP)  in  the  production  of  Air  Force  pilots.  In  the  design, 
implementation,  and  funding  of  any  training  program,  the  goal  is  to  maximize  the  performance 
of  the  program  for  a  given  budget.  The  conceptual  model  presented  in  this  paper  shows  that 
the  performance  of  the  Air  Force  Flight  Training  Program  (FTP),  which  is  composed  of  the 
FSP  and  Undergraduate  Pilot  Training  (UPT),  can  be  increased  at  a  reduced  cost. 

The  FTP  conceptual  model  was  composed  of  two  primary  components:  efficiency  and 
effectiveness.  Each  of  these  components  was  further  subdivided  into  two  subcomponents  of 
selection  and  training,  thus  creating  four  variables:  training  effectiveness,  selection  effectiveness, 
training  efficiency  and  selection  efficiency.  The  tree  diagram  in  Figure  1  depicts  the  FTP 
conceptual  model.  The  factors  in  parentheses  (e  g.,  FSP  graduation  rate)  represent  measures 
of  each  of  the  variables  selected  for  use.  Although  the  factors  serve  as  measures  for  their 
respective  variables  (as  shown  in  Figure  1),  Stoker,  Hunter,  Kantor,  Quebe,  and  Siem  (1987) 
showed  that  each  of  the  measures  also  has  a  secondary  effect  on  a  related  variable  (as 
indicated  by  the  dashed  lines  in  Figure  1).  For  example,  though  FSP  is  primarily  a  screening 
device  for  FTP  (and  thus,  a  measure  of  selection  effectiveness),  it  does  confer  some  training 
benefit  that  is  useful  in  UPT.  Conversely,  although  UPT  is  primarily  a  training  program,  it 
also  contributes  to  selection  effectiveness  in  that  it  eliminates  from  the  FTP  those  individuals 
who  are  not  qualified  for  further  flight  training. 


Flight  Training  Program  (FTP) 
Program  Performance 


(FTP  graduation  rate) 


Effectiveness 


( 


E4 


Selection 

Effectiveness 


Efficiency 
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Figure  1.  Conceptual  Model  of  Program  Performance  of  FTP. 
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Several  assumptions  about  selection  and  training  were  made  as  the  model  of  program 
performance  was  developed.  First,  it  was  assumed  that  both  selection  and  training  existed 
on  a  continuous  scale,  which  allowed  for  infinite  combinations  of  selection  and  training  This 
assumption  affected  the  various  relative  values  that  are  presented  later.  Next,  it  was  very 
difficult  to  separate  the  part  of  a  program  that  impacted  personnel  selection  and  the  part  that 
affected  training.  This  difficulty  and  the  lack  of  direct  measures  necessitated  the  use  of 
surrogate  measures  for  each  of  these  components. 

The  conceptual  model  of  the  FTP  will  be  presented  in  the  following  four  sections.  Section 
II  addresses  the  effectiveness  of  FTP  and  focuses  on  the  results  of  the  Stoker  et  al.  (1987) 
study.  It  also  shows  how  changes  in  the  amount  of  hours  flown  in  FSP  affected  the  FTP 
graduation  rate.  Section  III  on  efficiency  presents  a  cost  analysis  of  the  data  from  the  Stoker 
et  al.  report  and  some  possible  savings  if  alternative  training  recommendations  are  implemented. 
Section  IV  on  performance  brings  the  results  of  the  effectiveness  and  efficiency  sections  together 
through  the  use  of  indifference  curves.  The  concluding  section  outlines  recommendations  to 
improve  FTP. 


II.  EFFECTIVENESS 

The  two  variables  of  program  effectiveness  (PE)  used  in  the  model  were  selection  effectiveness 
(SE)  and  training  effectiveness  (TE).  SE  was  defined  as  the  ability  of  the  program  to  accurately 
select  those  candidates  who  have  the  aptitude  to  graduate  from  UPT,  and  used  FSP  graduation 
rate  as  the  measure  TE  was  defined  as  the  ability  of  the  program  to  prepare  and  train 
individuals  mentally  and  physically  so  that  they  can  be  successful  UPT  graduates  (Stoker  et 
al.,  1987);  the  measure  for  TE  was  UPT  graduation  rate. 

The  Stoker  et  al.  study  related  four  different  FSP  options  to  differing  FTP  graduation  rates. 
In  the  current  study,  results  from  Stoker  et  al.  were  used  to  construct  four  different  groups  of 
FTP  students,  with  each  group  experiencing  a  different  FSP.  Even  though  the  results  from 
Stoker  et  al.  form  the  basis  of  this  study,  they  are  used  here  only  as  a  demonstration  of  the 
methodology.  The  four  hypothesized  groups  used  in  this  study  are  described  below. 

Group  1.  This  group  would  not  experience  FSP.  Instead,  they  would  directly  enter  UPT 
with  no  prior  screening  or  training. 

Group  2.  The  students  from  this  group  would  enter  the  normal  1 4-hour  FSP  and  then  be 
selected  for  UPT  or  drop  out  of  the  program. 

Group  3.  These  students  would  enter  a  20-hour  FSP,  with  screening  taking  place  at  the 
same  time  as  that  experienced  by  Group  2.  The  6  additional  hours  of  FSP  would  provide 
more  training  for  the  students. 

Group  4.  This  group  was  not  directly  discussed  in  the  Stoker  et  al.  report.  However,  by 
introducing  the  Basic  Attributes  Tests  (BAT)--which  allow  for  screening  to  take  place  at  an 
earlier  time  with  an  equivalent  level  of  confidence-into  the  20-hour  program,  the  same  level 
of  screening  and  training  would  be  maintained  since  there  are  no  changes  in  the  flying  program 
of  FSP. 

Table  1  presents  a  summary  of  the  four  groups,  their  relationship  (if  any)  to  Stoker  et  al.. 
the  number  of  hours  flown  in  FSP  by  each  group,  and  their  respective  levels  of  SE  (FSP 
graduation  rate),  TE  (UPT  graduation  rate),  and  PE  (FTP  graduation  rate). 
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Table  1.  Result*  of  Differing  FSP  Options 


McGarrity- 

Curtis 

Group 

Stoker 
et  at. 
Group 

FSP  Option 
(Hours) 

SE 

(%) 

TE 

(%) 

PE 

(%) 

1 

1 

none 

100 

56 

56 

2 

3 

14 

77 

74 

57 

3 

2 

20 

77 

88 

68 

4 

- 

20  (+  BAT) 

77 

88 

68 

Although  Groups  1,  2  and  3  all  differed  from  each  other  by  the  amount  of  flying  time  that 
they  were  allowed,  Group  4  differed  from  the  other  groups  because  of  the  inclusion  of 

psychomotor  test  scores  from  the  Basic  Attributes  Tests  (BAT).  These  tests,  which  aided  in 
the  selection  decision  of  students  for  UPT,  measured  the  eye-hand  coordination  skills  of  the 
individual.  The  Stoker  et  at.  report  showed  that  selection  for  UPT  could  be  made  at  an  earlier 
point  in  FSP  (as  early  as  the  eighth  FSP  flying  hour)  using  flight  performance  data  and  results 
from  BAT  testing  and  be  as  statistically  accurate  as  a  selection  decision  made  ai  any  later 
time  without  BAT  results  (p.  10).  The  earlier  selection  decision  did  not  affect  SE,  PE  or  TE 

values  in  the  comparisons  between  Group  3  and  Group  4;  however,  it  did  have  an  impact  on 

the  efficiency  of  the  program  as  will  be  discussed  later. 

The  first  three  groups  had  differing  levels  of  SE  (FSP  graduation  rate)  and  TE  (UPT 

graduation  rate),  which  affected  the  overall  PE  as  measured  by  the  FTP  graduation  rate.  FTP 
rates  were  generated  by  multiplying  the  UPT  (TE)  and  the  FSP  'SE)  graduation  rates  together. 
SE  for  Group  1  was  assumed  to  be  100%  so  that  consistency  in  the  computations  could  be 
maintained.  The  SE  level  for  Group  1  provides  an  exampie  of  the  selection  and  training 
overlap  mentioned  earlier  since  all  selection  decisions  would  be  made  in  UPT  and  not  in  FSP. 
The  TE  level  of  the  first  group  was  56%  due  to  the  fact  that  the  UPT  graduation  rate  for  this 
group  was  56%. 

The  PE  value  (FTP  graduation  rate)  for  Group  2  increased  by  a  single  percentage  point 
over  that  for  Group  1,  due  to  an  increase  in  the  amount  of  TE  from  56%  to  74%.  Groups 
3  and  4  experienced  an  increased  level  of  PE  over  that  for  Group  2.  This  higher  level  (68% 
vs  57%)  was  a  result  of  the  increase  in  TE.  The  increase  in  PE  was  11  percentage  points 
higher  than  the  PE  for  Group  2  and  12  pf'centage  points  higher  than  that  for  Group  1. 

Figure  2  shows  how  increases  in  FSP  flying  hours  affect  FTP  graduation  rate.  The 
increasing  rate  of  the  slope  of  the  curve  suggests  that  at  some  point  beyond  the  20th  FSP 
hour,  a  more  effective  program  could  be  established  perhaps  because  more  skills  could  be 
taught  in  FSP  and  applied  in  UPT  than  is  currently  the  case.  The  lengthening  of  FSP  increases 
the  amount  of  training  that  would  go  on  in  FSP,  which  increases  the  overlap  between  the  two 
measures,  SE  and  TE. 


III.  EFFICIENCY 

To  determine  the  efficiency  component  of  Program  Performance,  a  costing  methodology  was 
developed  to  compute  the  cost  to  graduate  pilots  from  UPT.  By  specifying  that  the  number 
of  UPT  graduates  was  the  same  for  all  of  the  FSP  options,  it  was  possible  to  standardize 
the  results  and  provide  accurate  cost  comparisons.  As  in  effectiveness,  the  efficiency  measures 
were  split  into  variables  of  selection  and  training.  Selection  efficiency  (SEF)  was  measured 
by  the  cost  of  FSP,  and  training  efficiency  (TEF)  was  measured  by  the  cost  of  UPT. 
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Figure  2  FSP  Hours  vs  FTP  Graduation  Rate. 


As  an  example  of  the  cost  impact,  it  was  assumed  that  each  year  approximately  2.000 
UPT  graduates  are  required  by  the  Air  Force.  O'  these.  1.100  will  have  FSP  as  their  first 
introduction  to  flying.  (The  remaining  900  are  not  required  to  attend  FSP  because  they  have 
their  private  pilot  license  or  are  Air  Force  Academy  graduates  and  have  already  gone  through 
a  program  similar  to  FSP.)  Table  2  shows  the  various  rates  and  costs  for  the  differing  groups 
and  how  these  rates  affect  the  number  of  students  required  to  enter  FTP  in  order  to  generate 
1,100  graduates. 

The  values  in  columns  1,  4,  6.  and  7  cf  Table  2  are  Fisca1  Year  1985  data  from  Air 
Training  Command  (ATC).  The  UPT  graduation  rate  in  column  2  came  from  the  Stoker  et  al. 
study.  Column  3  was  calculated  by  dividing  column  1  by  column  2.  Dividing  column  3  by 
column  4  gave  the  required  FSP  entrants  in  column  5  Column  8  was  calculated  by  multiplying 
columns  5  and  6  together  except  for  Group  4.  (A  discussion  of  the  calculations  for  Group 
4’s  FSP  cost  is  In  the  next  paragraph.)  The  UPT  cost  in  column  9  was  calculated  by  multiplying 
columns  3  and  7  together.  The  total  cost  in  column  10  was  reached  by  adding  columns  8 
and  9.  Column  11  was  calculated  by  finding  the  difference  between  Group  2  s  cost  and  the 
cost  of  all  the  other  groups. 

In  that  Group  2  s  FTP  is  currently  being  used  by  ATC,  the  cost  of  Group  2’s  option  forms 
the  basis  from  which  the  other  groups  are  measured.  For  example,  to  compare  Group  2  (the 
existing  program)  and  Group  4  one  would  subtract  $283.4  million  (Group  4’s  cost)  from  $334.6 
million  (Group  2's  cost),  yielding  $51.2  million,  the  last  entry  in  column  11. 
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Table  2.  FS?  and  UPT  Costs  and  Rates 


Group 

1 

2 

3 

4 

5 

UPT 

Grads 

Stoker  et  al. 

UPT  grad 

Rate 

Required 

UPT 

entrants 

(1/2)* 

FSP 

grad 

rat* 

Required 

FSP 

entrants 

(3/4) 

1 

1100 

56% 

1965 

N/A 

N/A 

2 

1100 

74% 

1487 

;  '% 

1932 

3 

1100 

88% 

1250 

77% 

1 624 

4 

1100 

88% 

1250 

77% 

1624 

6 

7 

8 

9b 

10b 

11b 

FSP 

UPT 

FSP 

UPT 

FTP 

Savings 

cost / 

cost / 

cost 

cost 

C08t 

over 

Group 

entrant 

entrant 

(5x6) 

(3x7) 

(8+9) 

group  2 

1 

N/A 

$218637 

N/A 

$429.6 

$429.6 

-$95.0C 

2 

$4914 

$218637 

$9.5 

$325.1 

$334.6 

$0 

3 

$7020 

$218637 

$11.4 

$273.3 

$284.7 

$49.9 

4 

$2908/4212 

$218637 

$10.1d 

$273.3 

$283.4 

$51.2 

"Numbers  in  parentheses  represent  functions  between  columns  to  arrive  at  new  column  values  (eg.,  required 
number  of  UPT  graduates  In  column  1  is  divided  by  the  UPT  graduation  rate  in  column  2  to  derive  the  required 
number  of  UPT  entrants  in  column  3). 

b Costs  m  columns  8,  9,  10,  and  11  are  in  millions  of  dollars. 

CA  negative  savings  represents  increased  coat 
dSee  text  for  calculating  the  FSP  cost  for  Group  4. 


There  are  two  points  that  need  to  be  hignlighted  from  Table  2  concerning  the  FSP  cost 
estimation  for  Group  4.  First,  the  BAT  cost  associated  with  Group  4  was  computed  by 
multiplying  the  cost  to  administer  the  test,  $100,  times  the  number  in  column  6  of  Table  2  for 
Group  4  (Group  4  was  the  only  group  that  used  the  BAT  tests  in  the  selection  of  people  to 
go  to  UPT).  Next,  the  calculation  of  FSP  cost  for  Group  4  required  two  values  in  column  6 
of  Table  2.  The  calculation  is  different  from  the  one  shown  at  the  top  cf  the  column  because 
the  assumption  of  when  attrition  occurs  was  changed  in  the  fourth  group  from  that  in  the 
preceding  three  groups.  In  the  first  three  groups  it  was  assumed  that  attrition  occurred  at 
the  end  of  the  program;  for  Group  4,  attrition  was  assumed  to  occur  at  the  end  of  the  eighth 
hour  of  FSP.  This  created  two  sets  of  students  in  Group  4.  The  first  set  was  composed  of 
the  entrants  to  FSP,  and  the  second  set  was  composed  of  those  FSP  stude '  >  who  had  been 
selected  to  go  on  to  UPT.  Therefore,  the  cost  of  FSP  for  Group  4  was  calculated  by  multiplying 
the  number  of  students  in  column  5  by  the  first  cost  in  column  6  then  multiplying  the  number 
of  students  in  column  3  by  the  second  cost  in  column  6  and  adding  these  two  costs  together 
and  entering  the  result  into  column  8. 


IV.  PERFORMANCE 

The  conceptual  model  of  overall  FTP  program  performance  shown  in  Figure  1  requires  that 
program  effectiveness  and  program  efficiency  be  combined;  however,  this  was  not  possible 
because  program  effectiveness  and  program  efficiency  were  represented  using  different  metrics. 
To  correct  for  this  difference,  two  solutions  were  possible:  use  of  expert  opinion  or  application 
of  a  notional  representation.  Because  the  model  was  a  conceptual  one,  notional  representation 
was  selected  to  define  overall  program  performance  in  terms  of  effectiveness  and  efficiency. 
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To  show  the  relationship  between  efficiency  and  effectiveness,  the  economic  tool  of  indifference 
curve  analysis  was  selected.  An  indifference  curve  is  a  set  of  points  on  a  curve  which 
represent  various  combinations  of  goods  or  services  of  equivalent  utility  in  the  sense  that  each 
of  the  combinations  or  choices  would  be  of  equal  preference  to  a  decision  maker.  In  other 
words,  the  utility  of  one  group  of  goods  is  equal  to  the  utility  of  another  group  of  goods 
(Hirschleifer,  1984).  For  example,  referring  to  the  indifference  curves  shown  in  Figure  3,  a 
decision  maker  for  FTP  can  be  indifferent  between  a  program  consisting  of  a  specific  level  of 
efficiency  and  effectiveness  (point  A)  and  a  different  level  of  efficiency  and  effectiveness  (point 
B)  as  long  as  both  points  are  on  the  same  curve  (PP2). 
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Figure  3.  Effectiveness  vs  Efficiency  (All  Groups). 


An  improvement  in  training  or  selection  technologies  would  result  in  movement  to  another 
curve,  one  that  is  farther  from  the  origin.  (The  farther  a  curve  is  from  the  origin  in  a  positive 
direction,  the  greater  utility  to  the  decision  maker  the  curve  represents.)  Therefore,  the  mixture 
of  effectiveness  and  efficiency  at  point  C  in  Figure  3  is  preferred  over  the  options  represented 
by  points  A  and  B  since  point  C  lies  on  a  curve  (PP4)  which  is  farther  out  from  the  origin. 
Using  the  approach  of  indifference  curve  analysis  (Hirschleifer,  1984),  it  is  possible  to  demonstrate 
that  Group  4  in  this  study  had  the  highest  level  of  overall  program  performance. 

Curve  PP2  in  Figure  3  is  the  indifference  curve  for  all  mixes  of  efficiency  and  effectiveness 
having  equal  utility  for  Group  2  (the  xesent  FTP).  Group  2's  effectiveness  (57%  FTP  graduation 
rate)  and  efficiency  (no  dollar  savings)  are  plotted  as  point  B  on  PP2.  If  Group  4  had  the 
same  overall  performance  level  as  Group  2,  it  would  also  be  represented  by  PP2.  If  this 
were  the  case,  to  generate  the  higher  level  of  efficiency  shown  in  Table  1  for  Group  4  ($51.2 
million  savings)  and  remain  on  PP2,  there  would  need  to  be  a  drop  in  effectiveness  from  PEb 
to  PE*.  However,  this  did  not  occur.  Instead,  there  was  an  increase  in  PE  from  PEb  to  PEc 
(68%  graduation  rate),  which  was  possible  only  if  Group  4's  overall  performance  actually  lay 
on  a  PP  curve  having  greater  performance  or  utility,  in  this  case  curve  PP4.  By  using  similar 
logic,  it  can  be  seen  in  Figure  3  that  Group  4’s  program  performance  not  only  exceeded  that 
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of  Group  2,  but  was  greater  than  those  for  Groups  1  and  3.  By  utilizing  the  FTP  structure 
of  Group  4,  the  Air  Force  could  now  employ  an  FTP  which  is  more  effectiva  and  efficient  man 
that  currently  experienced- -without  the  extensions  to  the  program. 


V.  CONCLUSIONS  AND  RECOMMENDATIONS 

In  summary,  a  conceptual  framework  for  dealing  with  the  measurement  of  a  program’s 

success  was  established.  A  program’s  performance  was  shown  to  be  a  function  of  two 

components:  effectiveness  and  efficiency.  The  research  results  suggest  that  the  implementation 
of  a  program  similar  to  that  for  Group  4  (20  hours  of  FSP  flying  time,  and  the  use  of  the 
BAT)  would  save  the  Air  Force  approximately  $51.2  million  annually  and  provide  more  effective 
use  of  program  resources.  This  conclusion  is  based  on  a  comparison  of  different  program 
options  that  included  the  efficiency  and  the  effectiveness  of  each  in  producing  graduates  from 
FTP.  Indifference  curve  analysis  showed  that  a  change  in  training  and  selection 
procedures-through  the  use  of  the  BAT  and  an  improved,  20-hour  FSP  training  curriculum-resulted 
in  movement  to  a  new  performance  curve  that  represents  an  an  overall  higher  performance 
level  for  all  combinations  of  efficiency  and  effectiveness.  That  is,  as  time  spent  in  FSP 
increases,  FSP  becomes  less  of  a  selection  program  and  more  of  a  training  program.  Also, 

it  is  more  cost  effective  to  teach  basic  flying  skills  in  FSP  than  in  UPT.  In  summary,  an 

expanded  FSP  with  the  BAT  could,  at  a  low  cost,  substitute  for  some  of  the  costly  selection 
and  training  components  of  UPT. 
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